Reflections on Niyogi’s book ”The Informational Complexity of Learning”

Niyogi’s book entitled ”The Informational Complexity of Learning” [6] addresses the problem of learning from examples. He considers two distant fields: artificial neural networks and natural language. In both areas he gives a theoretical analysis of informational complexity, i.e. the effects of the size of the leaning set and the number of model parameters on the accuracy of learning depending on the target function class. After outlining the main ideas, this work discusses the usability of such results in practice and the relevance of the book in linguistic research, and also raises a philosophical question about the possibility of error prediction. 1 Outline of the Main Ideas Niyogi’s book entitled ”The Informational Complexity of Learning” [6] addresses the problem of learning from examples. He considers two distant fields: artificial neural networks and natural language. In this first section I will introduce the theoretical framework needed to understand the basic points of the book. Due to the lack of space I will avoid unnecessary technical details while trying to remain as clear and exact as possible. The problem of learning from examples is usually stated as the following function approximation problem. There is an unknown function f : X → Y from a function space F . Unlike f , F is supposed to be known. The problem is to find in Akusztikai Szemle IV(1–2) pp19–21, 2002, book review